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An Operating System Interface Method 



FIELD OF THE INVENTION 

This invention relates to a method for interfacing computer applications to a 
computer's operating system. 

BACKGROUND OF THE INVENTION 

The internal operational organization of today's computer is the culmination 
of numerous Simultaneous engineering considerations. These considerations have 
brought an ensemble of organizational tools to the task of developing and 
maintainine software constructed systems. Simultaneously, there is evolving in the 
computer environment mutual design considerations for computational efficiency. 
l0 systems security, functional modularity for increasing maintenance and 
compatibility, etc. 

One area, of .he presently accepted software architecture design, which has 
been held in a relatively static state, is the interfacing between a central operating 
svstem and peripheral applications programs. Given the present complex.* of 

l5 operatin* svstems and of applications programs, adding of functions to either an 
operating system or to an application program is a complex undertaking, wh.ch 
often requires slobal systems-scale testing procedures. There is a need m the art for 
a method of adding functions in a way that does not require such a cumbersome 
,evel of testine. Furthermore, there is a need in the art for a method of in.erfactng 

„ with an operating system in a way that does not contribute to the complex,™ of 
such testing. 
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SUMMARY OF THE INVENTION 

The present invention relates to an operating system interface method, for 
use in a systems-architecture having a processor for executing application programs 
or kernel modules. This method includes the steps of: providing a hook manager 
5 for managing hooking and unhooking of system calls by application programs or 
kernel modules; and passing control to at least one hook before a system call or 
after a system response. 

Operating systems are designed to run on a specific computer architecture. 
In the context of the preferred embodiment of the present invention, an operating 
o system is a substantially centralized software entity running on an externally 
monolithic aggregate of a computer and its associated peripheral devices (e.g. 
printer, display, etc.). In the context of other embodiments of the present invention, 
an operating system is a distributed ensemble of like or linked software entities 
strategically proliferated into a distributed processing architecture (e.g. a shuffle 
exchange, a hyper cube, an array processor, a client server network system, etc.). 

In each respective architecture, the operating system receives system calls 
from application programs or from kernel modules. Likewise, the operating system 
directs system responses to applications programs or to kernel modules. 
Accordingly, a hook manager is provided, by the present invention, to evaluate an 
aspect of each system call or system response. The evaluation takes each 
intercepted call or response, and either passes it on to its expected destination or 
diverts it to an alternative destination. 

By this evaluation and diversion, a general use interfacing with an operating 
system is provided in a way that does not contribute to the complexity of either 
operating system testing or applications program testing. 

From the vantage of the operating system, the manager is just another 
applications program. From the vantage of an applications program, the manager is 
the operating system. Most importantly, from the vantage of a software developer, 
the manager provides a convenient way to externally expand the services provided 
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by an opiating system, and ,o external* resolve incompatibilities between an 

application program and an operating system. 

Accordin e to one common scenano. features of an applications program are 

specifically built to conform to a special feature of an operating system. Thereafter, 
5 there is a desire to use this applications program with another operating system that 
does not include this special feature. According to the present inventton. the 
manaser intercepts all system calls. When the system call is for the special feature, 
the manager diverts the call to an external "special feature emulator". 

Accordine to another common scenano. there is a need to analysts the 
,„ efficiency of an operating system is a specific envirorament. According to the 
present invention, the manager d.verts a specific Cass of system calls to a call-log 
applicafon where these systems calls are recorded, and thereafter routed vta the 
manaoer to their original destination, the operating system. An apphcanons 
program is thereby provided data, e.g. the caU-log record, for the needed analvs.s. 
, 5 " Retumins to the issue of defining a -manager- with respect to a specfic 
computer architecture environment, the provided manager may be a smgle 
centralized emit* operating as an intermediary to the operatmg system of an 
externally monolithic aggregate of a computer and its associated peripheral devtces. 
For an operating svstem that is a distributed ensemble of like or linked software 
, entities strateeically proliferated into a distributed processing architecture, the 
manager may a like numbered ensemble of co-managers assigned one co-manager 

per operatina system entity. 

Not unlike human management models, the ' software" manager of the 
present invention may create and assign task specific, process specific, or call 
, 5 specific -supervisor- software entities to any whole or partial application program, 
or even to anv group of application programs. 

Those versed in the art should appreciate that ostensibly the present 
invention does no. provide any new funct.onahty to the overall computer 
hardware-softvvare conglomerate. Rather, the present invention allows for many of 
3 „ the known or desired functions to be incorporated into the conglomerate, m an 
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independent modular manner. This facilitates the upward compatibility of both 
operating systems and applications programs. This facility is especially cost 
efficient when operating systems or applications programs embody special features, 
as is commonly the case. Since it is never known in advance which of these 
5 features will survive as industry standards and which will lapse into oblivion, the 
present invention provides an efficient facile solution for those who have invested 
man-years into utilizing options destined for oblivion. 

Furthermore, the present invention provides a rapid development track for 
the marketing of new' system-like utility features, which have not heretofore been 

10 incorporated into operating systems. This rapid development track may prove to be 
useful to the developers of standard accepted operating systems, as a means for 
introducing new features and for testing their acceptance in the market. This rapid 
development track may also prove to be useful to small specialty-type developers, 
who provide peculiar emulation or conversion software routines to clients 

15 switching from one operating system to a different operating system. 

BRIEF DESCRIPTION OF THE DRAWINGS 

In order to understand the invention and to see how it may be carried out in 
practice, a preferred embodiment will now be described, by way of non-limiting 
example only, with reference to the accompanying drawings, in which: 
20 Fig. 1 is a block diagram of a systems-architecture; and 

Fig. 2 is a flowchart of a complete system call and system response cycle. 



DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT 

25 Fig. 1 shows a block diagram of a systems-architecture wherein a processor 

1 has an input modality interface 2 and an output modality interface 3. The 
processor includes an operating system 4. a hook manager 5. a set of application 
programs 6, and a set of kernel modules 7. 
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The present invention relates to an operating system interface method, for 
use in a systems-architecture having a processor for executing application programs 
or kernel modules. This method includes the steps of: 

(a) providing a hook manager for managing hooking and unhooking of. 
5 system calls by application programs or kernel modules; and 

(b) passing control to at least one hook before a system call or after a 
system response. 

According to an embodiment of the present invention, passing control 
includes copying additional information to at least one hook. Examples of 
10 "additional information*' include systems state information, process memory 
information, kernel tables, response status, etc. 

According to an embodiment of the present invention, hooking includes 
calling an application program interface (API) to receive system calls or to no 

longer receive system calls. 
. 15 According to an embodiment of the present invention, the hook manager 

includes an application program interface (API) allowing receiving of system 
responses or discontinuing receiving of system responses. 

According to an embodiment of the present invention, managing includes 

using the hook manager for: 
20 (a) building a chain of hooks in response to requests by an application 

program or kernel module to receive system calls; and 
(b) removing a hook from said chain of hooks, in response to requests by an 
application program or kernel module to no longer receive system calls. 

25 In the context of the present invention, a -chain- is an organization or 

structure such as a list, a stack, a tree, a net-like graph, etc. 

According to one variation of the present invention, building a chain of 
hooks is in sequential order based on when the requests are received. According to 
another variation of the present invention, removing a hook is in a 

30 non-last-in-first-out (non-LIFO) fashion. 
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According to an embodiment of the present invention, the hook manager is 
an application program or a kernel module or a part thereof. 

According to an embodiment of the present invention, passing control 
includes an interim step selected from the list: 
5 (a) a hook under management of a hook manager gaining access to the 

system call or system response; 

(b) logging the system call or system response; 

(c) filtering the system call or system response: 

(d) encrypting or decrypting the system call or system response: 

10 (e) compressing or decompressing the system call or system response: 

(f) modifying, or altering, or replacing the system call or system response; 
or 

(g) delaying the system call or system response. 

15 Fig. 2 shows a flowchart of a complete system call and system response cycle. 

Starting at a "BEGIN' 7 20, a query step "ARE THERE ANY PRE-CALL 
HOOKS?" 21 is imposed. If there are pre-call hooks, then "GET A HOOK FROM 
THE HOOK CHAIN ,? 22. "IF THIS IS THE END OF THE CHArN" 23. then 
make the original call at 24 to the operating system 4; otherwise perform optional 

20 tests or processing at 25 and return to 22. "Optional tests or processing'' may- 
include checking for permissions, setting a processing delay or priority, etc. 

Symmetrically, but starting from an operating system response to entry point 
30, initiates a post processing a query step "ARE THERE ANY POST-CALL 
HOOKS?" 31. If there are post-call hooks, then "GET A HOOK FROM THE 

25 HOOK CHAIN'" 32. L TF THIS IS THE END OF THE CHAIN" 33, then return to 
40; otherwise perform optional tests or processing at 35 and return to 32. Again, 
''optional tests or processing" may include checking for permissions, setting a 
processing delay or priority, etc. 
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The present invention relates to an operating system, interface method, for use 
in a systems-architecture having a processor for executing application programs or 
kernel modules, the method comprising the steps of: 

(a) intercepting a system call 21 made by an application program or by a 

5 kernel module; 

(b) in a hook chain, determining 22 which of the hooks is permitted to 
receive control before or after the system call; 

(c) if there is at least one determined hook, then selecting at least one of the 
determined hooks: and 

10 (d) passing control to the at least one hook of the selected hooks. 

The present invention also relates to an operating system interface method, 
for use in a systems-architecture having a processor for executing application 
pro 2 rams or kernel modules, the method comprising the steps of: 
, 5 " (a) intercepting a system response 31 made to an application program or to 

a kernel module: 

(b) in a hook chain, determining 32 which of the hooks is permitted to 
receive, control before or after the system response; 

(c) if there is at least one determined hook, then selecting at least one of the 
20 determined hooks: and 

(d) passing control to the at least one hook of the selected hooks. 

It should be appreciated that a system call and a system response are 
substantially equivalent with respect to functionality and complexity, in the context 

^5 of the present invention. 

In the context of the present invention, the expression "determined hook- 
relates to the notion "determining using an algorithm, as ,s commonly used m 
operating systems security, scheduling, etc. For example, determining to assrgn, 
verify, or certify a most recent state of, status of, modification of. or systems 

;o characterization of a system related user. Hie. or resource. 
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According to an embodiment of the present invention, intercepting 
includes providing a hook manager. Furthermore, according to one variation of the 
present invention, the hook manager includes an application program interface 
5 (API). According to another variation of the present invention, the hook manager is 
an application program or a kernel module or a part thereof 

According to the preferred embodiment of the present invention, passing 
control includes an interim step selected from the list: 

(a) a hook under management of a hook manager gaining access to the 
0 system call or a system response; 

(b) logging the system call or a system response; 

(c) filtering the system call or a system response; 

(d) encrypting or decrypting the system call or a system response; 

(e) compressing or decompressing the system call or a system response; 

(f) modifying, or altering, or replacing the system call or a system response; 
or 

(g) delaying the system call or a system response. 

Furthermore, the present invention relates to an operating system 4 
interface-apparatus comprising systems-architecture having a processor 1 for 
executing application programs 6 or kernel modules 7, wherein said 
systems-architecture includes : 

• a hook manager 5 for managing hooking and unhooking of system calls 
by application programs or kernel modules; and 

• a controller (in 1 or 4) for passing to at least one hook before a system 
call or after a system response. 



In the method claims which follow; alphabetic characters used to designate 
claim steps are provided for convenience only and do not imply any particular order 
of performing the steps. 
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CLAIMS: . 

I An operate system interface method, for use in a systems-architecture having 
a processor for executing application programs or kernel modules, the method 

comprising the steps of: 

(a) providing a hook manager for managing hooking and unhooking of 
svstem calls by application programs or kernel modules; and 

(b) passing control to at leas, one hook before a system call or after a 

svstem response. 

2. The method according to claim 1 wherein passing control includes copying 
,0 additional information to at least one hook 

i - . i ^-hi-rein hookins includes calling an 

3. The method according to claim 1 wherein nooKin = 

application program interface (API) to receive system calls. 

4. The method according to claim 1 wherein unhookmg includes calling an 
application program interface (API) to no longer receive system calls. 

„ 5 The method according to claim 1 wherein the hook manager includes an 
" application prozram interface (API) allowing receiving of system responses. 

6 The method according to ciaim I wherein the hook manager includes an 
application program interface (API) discontinuing receiving of system responses. 
7. The method according to claim 1 wherein managing inc.udes using the hook 

20 manager for: . 

■ (a) building a chain of hooks in response to requests by an application 

prooram or kernel module to receive system calls; and 
(b) removing a hook from said chain of hooks, in response to requests by an 
application program or kernel module to no longer receive system calls. 
25 8. The method according to claim 7 wherein building a chain of hooks is m 
sequential order based on when the requests are received. 

9. The method according to claim 7 wherein removing is in a non-last-in-first-out 
(non-LlFO) fashion. 

. • , wherein the hook manager is an 

10. The method according to claim 1 wherein me 

30 application program or a kernel module or a part thereof. 
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interim step selected from the list: 

(a) a hook under management of a hook manager gaining access to the 
system call or system response; 
5 (b) logging the system call or system response; 

(c) filtering the system call or system response; 

(d) encrypting or decrypting the system call or system response; 

(e) compressing or decompressing the system call or system response; 

(0 modifying, or altering, or replacing the system call or system response; 
o or 

(g) delaying the system call or system response. 

12. An operating system interface method, for use in a systems-architecture 
having a processor for executing application programs or kernel modules, the 
method comprising the steps of: 

5 (a) intercepting a system call made by an application program or by a kernel 

module; 

(b) in a hook chain, determining which of the hooks is permitted to receive 
control before or after the system call; 

(c) if there is at least one determined hook, then selecting at least one of the 
o determined hooks; and 

(d) passing control to the at least one hook of the selected hooks. 

13. The method according to claim 12 wherein intercepting includes providing 
a hook manager. 

14. The method according to claim 13 wherein the hook manager includes an 
application program interface (API). 

15. The method according to claim 13 wherein the hook manager is an 
application program or a kernel module or a part thereof. 

16. The method according to claim 12 wherein passing control includes an 
interim step selected from the list: 
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(a) a hook under management of a hook manager gaining access to the 

system call or a system response; 

i 

(b) logging the system call or a system response; 

(c) filtering the system call or a system response; 

5 (d) encrypting or decrypting the system call or a system response; 

(e) compressing or decompressing the system call or a system response; 

(f) modifying, or altering, or replacing the system call or a system response; 



15 



or 



(g) delaying the system call or a system response. 



17. An operating system interface method, for use in a systems- architecture 
having a processor for executing application programs or kernel modules, the 
method comprising the steps of: 

(a) intercepting a system response made to an application program or to a 

kernel module; 

(b) in a hook chain, determining which of the hooks is permitted to receive 
control hefore or after the system response; 

(c) if there is at least one determined hook, then selecting at least one of the 
2 0 determined hooks; and 

(d) passing control to the at least one hook of the selected hooks. 

18. The method according to claim 1 7 wherein intercepting includes providing 
a hook manager. 

19. The method according to claim 18 wherein the hook manager includes an 
25 application program interface (API). 

20. The method according to claim 18 wherein the hook manager is an 
application program or a kernel module or a part thereof. 

21. The method according to claim 17 wherein passing control includes an 
interim step selected from the list: 
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(a) a hook under management of a hook manager gaining access to a system 
call or the system response; 

(b) logging a system call or the system response; 

(c) filtering a system call or the system response; 

(d) encrypting or decrypting a system call or the system response; 

(e) compressing or decompressing a system call or the system response; 

(f) modifying, or altering, or replacing a system call or the system response 
or 

(g) delaying a system call or the system response. 



22. An operating system interface-apparatus comprising systems-architecture 
having a processor for executing application programs or kernel modules, wherein 
said systems-architecture includes: 

(a) a hook manager for managing hooking and unhooking of system calls 
by application programs or kernel modules; and 

(b) a controller for passing to at least one hook before a system call or after 
a system response. 
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